Arc Post-Mortem Summary. Three parts covering wiki page visibility and site robots files. Part 1 fixed source and history visibility rules for wiki pages; Part 2 handled public page-list visibility; and Part 3 built per-domain robots.txt and humans.txt, covering their generation, serving, and the edit-page wiring.
Arc started 2026-06-16. Goal: let a group owner keep the public (and crawlers) out of wiki page source and history, and let each web-server domain publish its own robots.txt / humans.txt. Group-level CSS was considered and dropped — domain-level CSS already covers that need.
OPTIONS = (render_engine << 1) + encrypted,
which left no spare bit. Now:
SOCIAL_GROUPS.RENDER_ENGINE (fresh schema in
ProfileModel).OPTIONS holds
only yes/no flags: bit 0
GROUP_OPTION_ENCRYPTED, bit 1
GROUP_OPTION_PAGE_SOURCE_ALLOWED (new constants in
Config.php).GroupModel
pack/unpack updated: addGroup,
updateGroup, isGroupEncrypted,
getRenderEngine (reads the column),
getGroupById (selects the column, unpacks both
flag bits). Public method signatures unchanged, so callers do
not move.upgradeDatabaseVersion106
(DATABASE_VERSION 105→106): adds the column,
then in PHP (portable across all five PDO back ends) copies each
group's render engine out of OPTIONS and rewrites OPTIONS to
keep the encrypted bit and turn the page-source-allowed bit on,
so existing behaviour is preserved.PAGE_SOURCE_ALLOWED (the bit) controls public view of
both source and history. Default allowed. Set from a new "Public
Source & History" Allowed / Not Allowed dropdown in the group
edit form (ManagegroupsElement), wired through
PAGE_SOURCE_CODES and the
updateGroup field list.public_source head var now gates
history as well as source. Either the group flag or the page var
being off denies the public; editors and owners always see both.public_source is
off. NO_HISTORY_SOURCE also fires on the group flag, so
the Source and History links disappear from the page for the
public.public_source
checkbox is disabled and shows an "(off at group level)" note, so an
editor is not misled into thinking a page can opt back in.src/locale/en_US/configure.ini.NO_HISTORY_SOURCE signal (page var or group
flag off, owners exempt): the floated source/discuss buttons
(GroupbarElement), the page-mode menu's
source/history/discussion entries (GroupmenuElement),
and the page-listing "view source" button
(WikiElement, gated on the group flag).getPageInfoByThread.Arc folders for a shared day are now suffixed in the order they happened (2026-06-16A/B/C); existing 2026-06-16 arcs were relabelled accordingly. Past Z, suffixes double (AA, AB, …).
GROUP_OPTION_PAGE_LIST_ALLOWED (bit 2, value 4)
controlling whether the public may see a group's list of wiki pages.
Default allowed.upgradeDatabaseVersion107
(DATABASE_VERSION 106→107): turns the bit on for
every existing group in PHP, so the listing stays visible as
before.GroupModel packs
and unpacks the bit in addGroup, updateGroup
and getGroupById alongside the other flags.ManagegroupsElement), wired through
PAGE_LIST_CODES and the field list.GroupmenuElement). Members and owners
are unaffected.page_source_allowed and
page_list_allowed from the default group array, so
both keys are seeded (allowed) there to stop undefined-key warnings
on the Create Group page.Design agreed 2026-06-17 from Chris's spec. Each secure-route domain gets its own generated robots.txt and a free-form humans.txt, edited from the Web Server settings. Decisions: store in the per-domain appearance blob (no migration); a disallowed category blocks every address that reaches it (both the clean path and the query form); toggles are global for now (not per-group); the old shipped src/robots.txt is removed since this supersedes it.
RobotsTxtGenerator
library class (pure logic): turns a domain's saved yes/no choices
into robots.txt text. Disallow-all yields a blanket
Disallow: /; otherwise it opens for all crawlers, adds
a crawl-delay line, a Disallow block for every address leading to a
turned-off category (both clean path like /group and
query form like /*?*c=group), then the admin's extra
directives. An unconfigured domain defaults to everything crawlable
except search-result addresses.robots.txt to routeRobotsTxt and
humans.txt to routeHumansTxt. Each finds
the request host, reads that domain's saved config from the
appearance blob via a GroupModel, and emits generated
text through the WebSite object, exactly as the
mta-sts.txt route does.src/robots.txt; the generated default supersedes it.serverSettings&arg=robots&domain=....
CloseHelper [x] floated opposite; the
disallow-all checkbox (JS-hides the other robots inputs, humans
stays); the crawl-delay dropdown (1/3/5/10/30/60); the
search/feeds/wiki checkboxes with the three wiki sub-checkboxes
(source & history, page feeds, page lists); the additional-robots
and humans.txt textareas; and Save / Cancel / Preview (Preview opens
a tab with the generated robots.txt).serverSettings
handling: render the page for arg=robots, save the
choices into the domain's appearance blob (merging, so appearance
overrides are untouched), and a preview path that runs the same
generator.DOMAIN_APPEARANCE (the
per-domain serialized override table from the v104 migration).ServersettingsElement +
SystemComponent domain routes), each opening a simple
textarea editor.